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Abstract — We consider a multi-way relay network with an 
orthogonal uplink and correlated sources, and we characterise 
reliable communication (in the usual Shannon sense) with a 
single-letter expression. The characterisation is obtained using 
a joint source-channel random-coding argument, which is based 
on a combination of Wyner et al.'s Cascaded Slepian-Wolf Source 
Coding and Tuncel's Slepian-Wolf Coding over Broadcast Chan- 
nels. We prove a separation theorem for the special case of two 
nodes; that is, we show that a modular code architecture with 
separate source and channel coding functions is (asymptotically) 
optimal. Finally, we propose a practical coding scheme based on 
low-density parity-check codes, and we analyse its performance 
using multi-edge density evolution. 



I. Introduction 

CONSIDER a multi-way relay network in which a group 
of physically separated nodes exchange data. Direct 
communication between the nodes is not permitted, and the 
exchange is only made possible with the help of a relay. 
The nodes encode and transmit their data over an uplink (a 
multiple-access channel) to the Relay. The Relay processes 
this information and transmits over the downlink (a broadcast 
channel) to every node. We assume that each node requires a 
lossless reconstruction of the data of all other nodes. 

The above relay network aims to model communication in 
cellular and satellite networks. A large body of work has 
comprehensively studied the network from the perspective 
of source coding yj, channel capacity Q-Q, and network 
coding I?), |[8). However, despite this intense effort, the 
information-theoretic limits of the network remain largely 
unknown. 

We study the relay network under two specific assumptions. 
The first assumption is that the data is arbitrarily correlated 
- generated by a discrete-memoryless (DM) source - and the 
communications problem involves joint source-channel (JSC) 
coding. Correlated data might take the form of measurements 
in a sensor network [9J, voice data in a cellular network, and 
data files in a peer-to-peer network. We wish to determine 
when a given source can be reliably communicated (in the 
usual Shannon sense) over a given channel. 
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The second assumption is that the downlink is an arbitrary 
DM broadcast channel and the uplink is an orthogonal DM 
multiple-access channel. Our motivation to study an orthog- 
onal uplink stems from Shannon's classic separation theo- 
rem flO', Sec. 7.3], which states that the problem of losslessly 
transmitting a DM source over a point-to-point DM channel 
can be divided into two independent problems - source coding 
and channel coding. Moreover, the individual optimisation 
of stand-alone source and channel codes is optimal for the 
overall point-to-point JSC problem. The separation theorem is 
important in practice because, for example, systems are rarely 
restricted to transmitting a single source over a fixed channel 



with known statistics; indeed, to quote Gallager 1 1 1 Pg. 140] 
(see also fVl}.. Pg. 406]): 

"In many data transmission systems the probabilities 
with which the messages are to be used are either 
unknown or unmeaningful." 
The modular nature of the separate source-channel coding 
architecture allows the source and channel codes to be changed 
as needed, without compromising overall optimality |13|. 
Unfortunately, separation may or may not be optimal for 
networks in general; for example, separation is suboptimal for 
the multiple-access channel [10, Pg. 592] and the broadcast 
channel |14^ Sec. 14.2], and it is optimal for the orthogonal 
multiple-access channel fTS) . Given this state of affairs, it is 
natural to ask whether separation is optimal for the multi-way 
relay network. We prove, in this paper, that separation holds 
for the special case of two nodes. 

Paper Outline: 

• Section We formally define the JSC-coding multi- 
way relay problem, and we characterise reliable com- 
munication with matching single-letter achievability and 
converse theorems. The achievability proof employs a 
JSC random-coding argument, which builds on the virtual 
binning idea of Tuncel |16| and the cascaded Slepian- 
Wolf binning idea of Wyner et al. ||T|. 

• Section III : We formalise a notion of source-channel sep- 
aration, and we prove a separation theorem for two nodes; 
that is, it is asymptotically (in blocklength) optimal to 
separate the source and channel coding functions. 

• Section IV: We use the two-node separation theorem 
as a basis to design practical low complexity codes. 
Specifically, we consider source and channel codes based 
on low-density parity-check (LDPC) codes. We show 
how the individual LDPC codes for source and channel 
coding can be represented by a joint factor- graph |17l, 
and we use this graph to provide an alternative view of 
the separation theorem. Finally, we present a numerical 
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Fig. 1. The JSC-code architecture for the multi-way relay network with an orthogonal uplink. 



example and discuss the differences between joint and 
separate decoding. 

Notation: Random variables and their alphabets are identi- 
fied by uppercase and script letters respectively, e.g. W and 
W. Random vectors defined on the cartesian product of a set 
are identified by boldface font, e.g., 

W = {Wi,W2, . . . ,Wn) 

takes values from 

W = WxWx---xW. 
^ ^ > 

n 

Subsets and strict subsets of an alphabet are identified with C 
and C respectively. Set complement is denoted by a superscript 
c; e.g., if 

£C{1,2,...,L}, 

then 

£=^{l,2,...,i:}\£. 
If £ is a singleton, say C — {I}, then we write ^ I = {I}''- 

II. Joint Source-Channel Coding 

A. Setup 

Consider Fig. [T] Suppose that a discrete memoryless source 
emits an i.i.d. string 

{w^^\wr\ . . . , w'f), , . . . , . . . 

(W^^W^f,...,W^^'), (1) 

of arbitrarily distributed random variables {W-^' , W'-'^' , • ■ ■ , 
VF'^'). Let W'' denote the alphabet of the l-th random variable 
VF''' for each index / in {1, 2, ... , L}. The Vt^<'> -component of 
the sequence in ([TJ is given to Node /. The orthogonal uplink 
channel from Node I to the Relay is discrete and memoryless 
with input alphabet X'-^\ output alphabet 3^<° and transition 
probabilities 



The downlink broadcast channel is discrete and memoryless 
with input alphabet X'-"\ output alphabet 3^*'' at Node /, and 
transition probabilities 

g<"'(2/i, y2, . . . , ^ P[r« = 2/1, r<^' = ^2, 

A joint source-channel (JSC) code with blocklength n, see 
Fig.[l] is a collection of (2L + l)-maps: the encoder at Node /, 

0(0 . y^w _^ x^i). (2a) 
the encoder at the Relay, 

and the decoder at Node /, 

^(0 . yyw X ;>;(') — ^ VV<" x x • • • x W*"-'. (2c) 
Node I observes 

A (VK{'\1^<",...,W«) 

and transmits 

The Relay observes 

on the Z-th uplink channel, and it transmits 
Node / observes 

r«^(yf,r«,...,r«) 

and decodes 
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The average joint decoding error probability of a JSC-code 



IS 



for one or more I in {1, 2, . . . , L} 



(3) 



Definition 1: We say that reliable communication is achiev- 
able with JSC codes if there exists for each e > a code of 
the form Q with < £ for some sufficiently large integer n. 

B. Main Result 

The following notation is required for the next theorem. 



Denote the capacity 1 10 Eqn. 7.1] of the ^-th orthogonal uplink 
channel by 

CS^max/(X<";y<«-"), 
^ xm 

where the maximisation is over distributions for X''' on A"*''. 
If 

£ = {/l, ^2, • • • , ^|£|} 

is a nonempty subset of {1,2,..., L}, then let 

denote those random variables with indices belonging to C 
The next theorem is proved in Appendix [A] 

Theorem 1: Reliable communication is achievable with JSC 
codes if 

H{W'^^\W'^^'>) <Y,C% (4a) 

holds for each nonempty strict subset £ of {1, 2, ... , L}, and 
there exists a distribution for on A"*"' such that 



i?(w<~''|iy<") < /(x<">;r«) 



(4b) 



holds for each I in {1, 2, . . . , L}. Conversely, if reliable com- 
munication is achievable then (|4a]) or ( |4bj i hold as inequalities 
— instead of strict inequalities — for some 

C. Remarks 

Non-matched symbol rates: Theorem[T]characterises reliable 
communication for matched source and channel symbol rates; 
i.e., n source symbols are mapped to n channel symbols. The 
proof easily extends to the non-matched symbol rate setting 
where n source symbols map to m channel symbols. 

Networks of Channels: The uplink condition (|4a| closely 



resembles Han's generalisation 1 15 Sec. 1] of the Slepian 



Wolf/Cover theorem fTSi to networks of noisy orthogonal 
channels (see also Barros and Servetto |jl9J). Indeed, all but 
one of the inequalities appearing in p3] p. 1] also appear as 
uplink constraints in (Hal — the exception being a total sum 



rate constraint of the form 



up- 



(5) 



1=1 



Although our problem formulation differs from that of p3) , 
[[T9), the similarity of these results can be understood by 



comparing the respective achievability proofs. Han fTS] uses a 
simple separate source-channel coding argument: he combines 
an optimal Slepian- Wolf code with optimal channel codes for 
each orthogonal uplink. In Han's setup, reliable communica- 
tion is possible if ( |4a] i and (|5]l both hold. The uplink part of 
our proof essentially uses the same argument, except we do 
not require that (|5]l holds; i.e., we use fewer bins and, as a 
consequence, the Relay cannot decode the sources. Indeed, in 
our setup, the Relay needs only to recover the Slepian- Wolf bin 
indices and not the individual source sequences. The downlink 
achievability proof requires JSC coding and is discussed next. 

Joint Source-Channel Coding: The (downlink) achievability 
proof of Theorem [T] is based on a JSC random-coding argu- 
ment that builds upon the virtual binning idea developed by 
Tuncel in fTF|. To illustrate why the virtual binning approach 
is useful, momentarily suppose that the Relay is given the 
entire source L-tuple {W-^\ W'^^\ . . . , W^^'') and consider the 
downlink phase in isolation. With the setup of |16| in mind, 
we can view (W'^', VF*^', . . . , PF*^') as a common message 
that needs to be rehably decoded by every node. Applying 1 16 
Thm. 6] we immediately see that the common message can 
be reliably decoded by every node whenever ( |4b] l holds. 

Thm. 6] is to 



The basic idea behind the proof of 1 16 



randomly generate a downlink channel codeword {n i.i.d. sym- 
bols ~ Px(0)) for each and every jointly typical source tuple 
(W'^', W'^\ . . . , W'^'). Upon observing a typical source tu- 
pl^ the Relay transmits the corresponding channel codeword. 
Node I, upon observing the channel output 1^''^ compiles a 
list of all those channel codewords that are jointly typical 
with 1^"'. The codeword list corresponds to list of typical 
source sequences, with the same number of elements. We 
may think of the source list as a (virtual) random bin in the 
sense of the classic Slepian- Wolf Theorem f20|. Node I looks 
within this list for a unique source tuple that is jointly typical 
with its source W'-'^; this search will be successful with high 
probability whenever (|4b]) holds. 

The above argument assumes that the entire source tuple is 
made available to the Relay, which is not the case in the multi- 
way relay network. The key difficulty in proving Theorem [T] is 
to overcome the fact that the Relay only has partial knowledge 
of the source tuple. 

Processing Broadcast Satellite: The source coding work of 
Wyner et al. [1 Thm. 1] is a special case of Theorem [T] 

III. Separate Source-Channel Coding 

We now compare the general JSC coding architecture of 
Section [n] to a separate source-channel coding architecture. 

A. Channel Coding 

The channel-coding problem of interest is analogous to the 
JSC-coding problem in Fig. [T] with one exception: the discrete 
memoryless source {W^''\ W^^\ . . . , W*^') is replaced by L- 
independent random variables (M'^\ M<^', . . . , M<^'), where 
each M''^ is uniformly distributed on Ai*-''. 

'An en'or is declared if tlie source is not jointly typical. 
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A channel code with blocklength n is a collection of maps: 
the encoder at Node I, 



1(0,1,) 



the encoder at the Relay, 

^(0) . yio.i) ^ yM X ..... ^ 

and the decoder at Node I, 

^Z-'" : 7V4<" X — > X<^> X x 



X M'' 



(6a) 



(6b) 



(6c) 



The channel code operates in a manner analogous to the JSC- 
code; Node I sends 

and decodes 

The average joint error probability of a channel code is 
defined by 

for one or more Z in {1, 2, ... , L}] . 

The rate at which Node / transmits the message M<'' is defined 
by 

,„ . 1 



log, \M^' 



Definition 2: A nonnegative rate tuple (r<^' , r<^' , . . 



>) is 



said to be achievable if the following holds: for each e > 
there exists a channel code of the form (|6]l with Pe < e and 
jy(o > j,(0 _ £ for some sufficiently large integer n. 

Definition 3: The capacity region C is the set of all achiev- 
able rates. 

We now give a single-letter expression for C. Let C* denote 
those nonnegative rate tuples (r<^' , r<^' , . . . , ) for which 

(i) the uplink channel capacities satisfy 

r« < C« 

for all I — 1,2, . . . , L; and 

(ii) there is a distribution for X'"^ on A"'"' such that 

^r<''' < /(X'"';y<'>) 



holds for alU = 1,2, . 
Lemma 1: C = C* . 



Proof: The lemma can be proved in the same way 
as Theorem [l] with (VF™,VF<", . . . ,W^'''>) replaced by 
(A'/«,M<"',..., A'f<^)) and H{W"'^\W"''^) replaced by 
J2iec ^® technical details. ■ 



,(1) 



,(1) 



(VK"'",...,W<^'^') 



,(2) 



Fig. 2. The multi-way relay network source coding problem. The source code 
is designed on the assumption that the L-indices (Af*^', A/*^', . . . , Af*^') can 
be reliably transported over the network by a channel code. 



B. Source Coding 

The source-coding problem of interest is the multi-source 
multicast problem shown in Fig. |2] The problem is the 
source coding counterpart of the channel coding problem of 



Section III-A in the following sense: a source code from this 
section combined with a channel code from the Section UlI-AI 
produces a (separate source-channel) code for the overall JSC 
problem. 

A source code of length n is a collection of 2i-maps: the 
compressor at Node I, 



0s : ~^M^'^; 
and the decompressor at Node I, 



V's : X X<^> X M'- 



' X ■ ■ ■ X M. 



X W'^ 



(7a) 



(7b) 



Node I sends 



and decompresses 



The joint decoding error probability is defined in the same 
way as ([3]), and the compression rate of Node I is defined by 



,(1) A 



1 



log2 \M'' 



Definition 4: A nonnegative rate tuple [r^^' , r^> , 



)) is 



said to be achievable if the following holds: for each e > 
there exists a source code of the form (|7]i with n sufficiently 
large, Pe < e and rjf < r<" + e for alU = 1, 2, . . . , L. 

Definition 5: The source-coding rate region TZ is defined 
as the set of all achievable rate tuples. 
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Lemma 2: TZ is equal to the set of all nonnegative rate 



tuples (r'' 



^ 7-<^) ) for which 



holds for each nonempty and strict subset £ of {1,2, . . . , L}. 

Proof: The lemma is a simple consequence of the Slepian- 
Wolf/Cover theorem {TH, ||2l|. The details are omitted. ■ 

C. Separate Source and Channel Coding 

Reliable communication with separate source and channel 
codes is possible if the intersection of the interior of TZ and 
the interior of C is nonempty. 

Theorem 2: Reliable communication with separate source 
and channel codes is possible if there exists nonnegative rates 
(rm,r<"',...,r<^>) such that 

i/(M/<^) < iM < (^w (8a) 

kec lec 

holds for each nonempty and strict subset £ of {1,2,..., L} 
and 

^r"-' </(X<'";y<") (8b) 

holds for all I in {1,2, ...,L}. 

Proof: The theorem is an immediate consequence of 
Lemmas [T] and |2] ■ 

Separate source and channel coding is optimal for two 
nodes. Specifically, the achievability assertion of Theorem |2] 
is equivalent to that of Theorem [T] if (j4| holds, then we can 
find (r'^' , r'^' ) simultaneously satisfying 

and 

The situation is more complicated for three or more nodes. 
Indeed, the achievability assertion of Theorem |2] is more 
restrictive than that of Theorem [T] in general; in particular, 
it is not possible to simultaneously remove all redundancies 
in the sources for every node, and such redundancies can be 
exploited by the channel code. We describe such a situation 
in Appendix |B] 

It should be noted that the capacity region C is formulated 
with the requirement that each node reliably communicates 
a single message to every other node. A more general setup 
would permit the use of a private message from each node to 
each subset of nodes. Characterising the resultant L(2^^^ — 1)- 
dimensional capacity region appears to be a formidable task; 
in particular, the problem includes the setup of |22| as a 
special case. Theorem [2] should therefore be understood as 
a sufficient condition for separate source and channel coding 
to be optimal. Finally, it is interesting to juxtapose such 
difficulties to the relatively simple JSC coding scheme used 
to prove Theorem [T] 



IV. Practical Codes 

We now consider the problem of designing practical, low 
complexity, codes that can approach those theoretical limits 
established for JSC coding in Sections |ll] and III Iterative 
error correction codes have been extensively investigated for 
JSC coding; for example, see f23l on joint turbo decoding and 
estimation of hidden Markov sources, or |24| on distributed 
JSC coding of video. In this section, we present a JSC coding 
scheme for the two-way relay network that is based on low- 
density parity-check (LDPC) codes |j25)-f28 |. In particular, we 
consider multi-edge LDPC codes [28 1 that are widely used in 
applications such as wiretap and multi -relay channels \29], 
|30|. Our aim is to show how the individual codes for source 
and channel coding can be represented by a joint factor-graph 
1 17], and we use this graph to provide an alternative view of 
the separation principle using multi-edge density evolution. 

In the following, we represent the channel coding message 



of Node I (denoted by Af in Section III-A i using the binary 
notation B*'' . As there are only two nodes, we will denote the 
node that is not Node I as Node ^ I. 

The encoder at Node I uses a linear source code to compress 
its source vector W'' as 



Here W'' and B<'' are binary vectors of length n and r'^'n 
respectively, and Ef is the parity-check matrix defining the 
source code of Node I. 

The compressed vector B''' is mapped to a channel code- 
word X''', which is transmitted over the uplink channel. The 
uplink channel code is defined by a parity-check matrix H^'j^ 
The relay decodes B''' from the noisy outputs Y'" of the ^-th 
uplink channel. It then maps the concatenation of B'^' and B<^* 
to X<°' using a channel encoder defined by the parity-check 
matrix EfK Each node uses a channel decoder to recover the 
other user's index B'~'' using their own index B''' and the 
noisy observation Y<'' from the broadcast channel. Finally, 
the source code is decoded separately resulting in an estimate 
of W~'\ This system is represented by the factor-graph in 
Fig. [3] 

A. Source-Channel Separation from a Factor-Graph View- 
point 



In Section III-C we showed that separate source-channel 



coding is asymptotically (in blocklength) optimal in the two- 
way relay network. We now provide an alternative view of 
this result using the factor-graph representation in Fig. [3] Let 
us first assume that the relay successfully decodes each bin 
index - the uplink code does not add to the discussion on 
separate versus joint decoding of the source and downlink 
codes. Consider the graph shown in Fig. [3] depicting the source 
codes and downlink channel code; these codes are connected 
via the bin indices. 

A separate source-channel decoder will apply the channel 
decoder to determine the other user's index B<~'' from Y<'* 

-For more information on liow to encode a message B*'' into a codeword 
X''' for a particular code hJ.'' see [28. App. A]. 
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Fig. 3. Factor-graph of the overall system. Circles represent (binaiy) symbols; parity-check equations are represented by solid black squares. The correlation 
between the symbols and the channel transition probability for the downlink are represented by empty squares. 



and then separately apply the source decoder to estimate 
W'~'' from B<~''. A joint source-channel decoder will decode 
W'~'* directly from Y<'' by decoding the source and channel 
codes on the joint factor graph, and exchanging soft (extrinsic) 
information between the two parts of the factor graph. 

Consider now the case when the source is compressed with 
a rate that equals the conditional entropy. In the channel coding 
setting this corresponds to a capacity achieving code. It has 
been shown in [31 J that the extrinsic information about the 
coded bits of any good (capacity achieving) code is zero above 
capacity. This implies that a joint decoder cannot outperform 
a separate decoding scheme in such a setting since there is no 
extrinsic information available to the joint decoder. Therefore, 
source-channel separation with separate decoding can be seen 
to be optimal from a factor graph perspective if capacity 
achieving source codes are applied. 

In the following we will investigate source-channel separa- 
tion in a practical setting by designing source and channel 
codes for the two-way relay channel and considering their 
performance using both density evolution and finite length 
simulations. 

B. Design of the Source and Channel Codes 

The task of the source code is to map the message W^'^ of 
length n bits to a message B^'^ of length r<'>n bits such that the 
other node can reconstruct the message using its own message 
as side information. This is a Slepian-Wolf coding problem 



with bin index B*^'). An optimal Slepian-Wolf code can 
be realised by using the syndrome of a linear code, which 
is optimised for a particular symmetric dual channel p2) , 
|33|. The optimisation of the degree distribution of an LDPC 
code for a symmetric channel with uniform input is well 
studied p8) , p4| and we will design our source codes in 
this way. 

For the downlink the relay has to communicate the messages 
B<^' and B'^' to both nodes simultaneously by broadcasting a 
codeword X<°'. Our proposed coding scheme is as follows. 
First, the relay treats the concatenation of B<^* and B'^' as 
the systematic part of an LDPC code. It then uses an LDPC 
encoder to determine the message X'°' (the "parity" bits), 
which it transmits over the broadcast channel. At each node, 
the channel decoder knows its own message B<'' and treats the 
message of the other node as being erased. 

The overall structure of the parity-check matrix H<,°' of 
our scheme is shown in Fig. [4] This matrix consists of 
a concatenation of the matrices H'^\ H'^' (corresponding to 
B<i', B(">), and the matrix tf"' (corresponding to X<"0. The 
codebook used by the relay is therefore defined as 

: [b«b<^'x<"']H<°''^ = o}. 
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Fig. 4. Structure of the parity-check matrix for the downlink channel code. 



The relay can determine its broadcast message X*"' by solving 
where ® denotes the element- wise addition over GF(2). An 



efficient algorithm to determine X'"' is described in |28 App. 
A]. 

To allow successful encoding at the relay and decoding at 
the nodes: 

• the square matrix H*"* has to be of full rank (over GF(2)) 
(this enables the relay to determine the vector X<"' given 
the messages B'^' and B<^'), 

• the matrices H'^' and H'^' have to be free of stopping sets 
(a set of variable nodes that, if erased, cannot be resolved 
by a message-passing decoder |j28j Sec. 3.22]). 

The first condition is obvious since the relay cannot determine 
its broadcast message if the square matrix H<°' is rank deficient. 
To show the necessity of the second condition, assume that 
the broadcast channel is noiseless, i.e., the nodes know X'"' 
without errors. The nodes now consider their own message 
B<'' as being known and solve for the other message B'~'' 
that is treated as being erased. If this set of erased variables 
contains a stopping set, then the iterative decoder is unable 
to solve for all erasures and gets stuck in the largest stopping 
setE] 

The absence of stopping sets can also be expressed by 
requiring that the sub-matrices H'^' and H'^> can be converted 
to triangular form using only row and column swaps. This 
corresponds to the encoding problem of LDPC codes as 



described in detail in |28 App. A] which allows us to apply 
their results. In particular, we use p28^ Thm. A. 16] to design 
LDPC codes which can be converted to triangular form. 

An optimised code for the downlink phase has to be 
designed for both nodes simultaneously, i.e., it has to perform 
close to the respective theoretical limit for both downlink 
channels. Such a code can be analysed and optimised using 



multi-edge type density evolution |28 Sec. 7]. 



^^We note that this constraint only applies to the (sub-optimal) message 
passing decoder which we consider in this paper For a maximum likelihood 
decoder we would have to require that the erased variable nodes do not contain 
the support of a codeword. The support of any codeword is a stopping set but 
not all stopping sets correspond to codewords. Therefore, the constraint for 
the message passing decoder is more restrictive than that for the maximum 
likelihood decoder. 



C. Density Evolution and Numerical Examples 

We apply multi-edge type density evolution to analyse the 
performance of the coding structure introduced in the previous 
section. We make the following assumptions for our numerical 
examples: 

• {W'-^\W''^^) is a doubly symmetric binary source with 
cross-over probability < p < 1/2, i.e.. 



(1/2) -(l-p) ifw^' 



(1/2) • p otherwise. 
The conditional entropy is therefore 

i7(W^(i)|M/(2)) ^ i/(M/(2)|T47(i)) = h{p), 

where h{-) denotes the binary entropy function. 

• The (orthogonal) uplink channels are binary input addi- 
tive white Gaussian noise (BIAWGN) channels with noise 
variance <t^ each. 

• The downlink channels are modelled as BIAWGN chan- 
nels with noise variance (t| each. 

Furthermore assume that the noise variance of the uplink 
channel is fixed and the nodes can communicate their mes- 
sages reliably to the relay. For the downlink we are interested 
in the pairs {p, cr^) which define the achievable region 



'D=Up, <7i) e (0, 1/2) X M+ : lim Pqe W <e}, (9) 

£— >oo J 

where e > is an arbitrarily small constant and Pde(^) 
denotes the bit error probability of the estimate W'~'' after 
£ iterations of message passing decoding, i.e., V consists of 
all pairs of source correlation p and noise variance <tJ on the 
downlink channel where the iterative decoder converges to an 
arbitrarily small error probability for sufficiently large block 
length. 

Since we assume that the uplink channels can be decoded 
by the relay, we focus on the decoding problem at the nodes 
where we have the constraint: 



Hp) < r« < Cbiawgn{<jI) 



(10) 



where CbiawgnIc^) denotes the capacity of a BIAWGN 
channel with noise variance a^. 

1) Example 1 (Separate Decoding): The decoder at Node I 
first decodes the other node's bin index B'~''. After performing 
a hard decision on the bin index, the node decodes the Slepian- 
Wolf code H<~'' using the other node's bin index B*""" and its 
own message as side information. 

In this example we optimised source and channel codes for 
two cases 



r<" = r<^> = 1/2 and r<^' = r<^> = 1/4. 



The resulting source and channel codes are given in Table [l] In 
Fig.|5]we have marked the optimised points for both rates (by 
a cross and a square respectively). We can see that the (density 
evolution) performance of these optimised codes is close to the 
theoretical limits. Note that due to practical constraints, for 
example we considered only LDPC codes with a maximum 
node degree of 50, these codes are capacity approaching 
rather than capacity achieving and so their performance is still 
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TABLE I 

Degrees and degree distributions of all codes used for the numerical examples. See Fig.[3]for the definition of the degrees. 

Source codes Channel codes downlink 











separate 


separate 


joint 


rate 1/2 


rate 


1/4 


rate 


1/2 


rate 1/4 


rate 1/2 


degree i 




degree i 




degree i 




degree i 


^x.i 


degree i Xx^i 


2 


0.1710 


2 


0.1046 


2 


0.3657 


2 


0.3503 


2 0.5254 


3 


0.2075 


3 


0.1984 


3 


0.1203 


3 


0.0731 


3 0.1612 


8 


0.0800 


5 


0.1189 


12 


0.0963 


5 


0.0161 


18 0.1349 


9 


0.2657 


6 


U.0006 


13 


0.1797 


6 


0.2043 


19 U.1785 


47 


0.1864 


9 


0.1597 


45 


0.0162 


19 


0.0761 




48 


0.0894 


10 


0.0616 


46 


0.2218 


20 


0.0370 








19 


0.0458 






33 


0.1991 








20 


0.0453 






34 


0.0440 








24 


0.1881 
















25 


0.0770 














10 


dw 


22 


db 


3 


db 


3 


db 3 










dx 


4 


dx 


4 


dx 3 



bounded away from capacity, as shown by the small gap to 
the (1/2,1/2) and (1/4,1/4) points respectively. 

2) Example 2 (Decoding when the Source and Channel 
Vary): In this example we consider the codes from Example 
1 when the capacity of the downlink channel is higher and/or 
when the conditional entropy of the source is lower than the 
optimised values. In the case of separate source and channel 
coding this gives the rectangular regions shown in Fig. [5] 

We next consider the same source and channel codes we 
used in the rate- 1/2 case in Example 1, but now apply a 
joint decoder. This is achieved by applying the sum-product 
algorithm [17| to the joint factor graph consisting of the 
downlink code and the Slepian-Wolf source code, i.e., the 
graph shown in Fig. [3] assuming error-free uplink channels. 
The achievable regions for this joint decoder are shown in 
Fig. |6] and compared to the separate decoder (solid and dashed 
lines, respectively). 

When the conditional entropy of the source is less than the 
value for which the system was designed, the source code 
is of course no longer optimal. A separate decoder ignores 
the source when decoding the downlink channel code and 
therefore it cannot exploit any remaining redundancy after 
decoding the source. However, a decoder that jointly decodes 
the downlink channel code and the source code can exploit 
this remaining redundancy. The achievable region of the joint 
decoder is thus larger than that of the separate decoder due to 
its improved performance when the conditional entropy of the 
source is less than 1/2 at the same time that the capacity of 
the downlink channel is less than 1/2. 

The joint source-channel decoder can therefore be seen 
to be more robust to variations of the source and channel 
away from the design entropy and rate. So while separate 
decoding is optimal for the particular design rate/entropy (here 
1/2,1/2), joint decoding is more robust when the source and 
channel vary. Indeed, a close observation of Fig. |6] reveals 
that in this particular example the joint decoding scheme 
can decode successfully at lower downlink capacities than 
the separate scheme even for the (1/2,1/2) point where the 
separate scheme is optimised. This is because the designed 
source code is not capacity achieving (see the comment in 



Example 1) so it is not compressing at the theoretical limit 
and there is still some remaining redundancy to be exploited 
by the joint decoder 

3) Example 3 (Code Optimisation for Joint Decoding): In 
the previous example we presented the achievable region of 
a joint decoder but we used the same codes as in Example 
1, i.e., codes that have been optimised for separate decoding 
of a specific source. Now we use the same source code as 
above but instead of optimising the channel code for the 
downlink for one particular source we optimise it for a range of 
sources. In particular, we chose to minimise the area between 
the achievable region and the theoretical limit over the entire 
range of source entropies. The results of such an optimisation 
process are shown in Fig. |6] (dash-dotted line). We observe that 
such an optimised scheme performs close to the limit over a 
wide range. However, this comes with a loss (at the rate 1/2 
point) compared to a coding scheme which is optimised for 
that particular setting. 

4) Example 4 (Finite Length Results): In addition to the 
asymptotic results of the previous examples we present finite 
length results. For this purpose we consider the codes of 
Examples 1 and 2, i.e., codes of rate 1/2 which have been 
optimised for a separate decoder. Finite length codes have 
been constructed for source blocks of length n = 10'' using 
the progressive edge growth (PEG) algorithm |35|. For the 
simulations shown in Fig.|7]we fixed the cross-over probability 
p of the source and varied the signal-to-noise ratio (Es/Nq) of 
the downlink channel (on the horizontal axis). We repeat this 
process for three separate sources (p — 0.09, 0.07 and 0.05). 
Results are shown for a separate and joint decoder (solid and 
dashed lines, respectively). 

First, consider the case where p — 0.09. This cross-over 
probability is close to the threshold of the source code (pth = 
0.1064) which leads to a high probability of error of the finite 
length source code (at approximately 10^^). These errors are 
independent of whether a separate or joint decoder is used and 
are because the finite length source code is far from capacity 
achieving. However, since the source cross-over probability is 
still slightly below the threshold, a joint decoder can exploit 
the remaining redundancy and can decode at a slightly lower 



0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

H(W'-''>\W^~'^] = h{p) 

Fig. 5. Achievable pairs of p and cr^ in terms of h{p) and CbiawgnC""^) for 
rates 1/2 (dashed line) and rates 1/4 (solid line). The dotted line represents 
the theoretical limit jlQ) . The uplink from the nodes to the relay is assumed 
to be error-free. 



0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

H(W^'1\W'~''>) = hip) 

Fig. 6. Achievable pairs of p and cr^ in terms of h{p) and CbiawgnC'T'^) 
for rates 1/2. The dashed line corresponds to a separate decoder (Example 
1) and the solid line corresponds to a joint decoder operating on the code 
designed for a separate scheme (Example 2). The dash-dotted line represents 
a joint decoder where the channel code is optimised to perform well over a 
wide range of sources (Example 3). 



Es/Nq for the downlink channel at bit error rates above this 
error floor. 

The error floor caused by the source code is decreased 
when the sources have a smaller cross-over probability (i.e. 
the sources have a greater correlation). For example, lowering 
the cross-over probability to p = 0.07 oi p = 0.05 shown in 
Fig. |7] leads to an error probability of the source code which 
is below our simulation range. In these cases a joint decoder 
will exploit the remaining redundancy and is able to decode 
at a significantly lower Es/Nq for the downlink channel than 
the separate decoder. The channel coding part of the separate 
decoder cannot benefit from a lower p and so has the same 
downlink channel performance in all three cases. 

This leads to the conclusion that, while source/channel 
separation is optimal in an asymptotic setting of infinite block 
length, any practical finite length system will benefit from a 
joint decoder 

Appendix A 
Proof of Theorem[T] 

In the usual way, we split the proof of Theorem [T] into two 
parts: the achievability assertion and the converse assertion. 

A. Proof of Theorem [7] (Achievability) 

1 ) Proof Outline: The main elements of the proof are best 
understood for the special case where the uplink channels 
are noiseless. Extending the proof to the more general case 
claimed in Theorem [T] is straightforward. 



The (noiseless) uplink random-coding argument uses a 
Slepian- Wolf/Cover distributed source code, e.g. fl4\ Sec. 10]. 
The source sequence at each node is compressed to a bin index 
that is sent to the Relay over the noiseless uplink channel. 
The L compression rates of the source codes are selected to 
satisfy all but one inequality in the Slepian-Wolf Theorem p4] 
Thm. 10.3] - the exception being the total sum rate inequality, 
e.g. (|5]l is omitted. The Relay has direct access to the L bin 
indices; it does not attempt to decode the individual source 
sequences. The (noisy) downlink random coding argument 
combines the virtual binning idea of Tuncel | [T6| with the 
cascaded Slepian-Wolf binning idea of Wyner et al. fT]. Each 
node will use a JSC-decoder to recover the source sequences. 

2} Preliminaries: We use typ/ca/ se^'Mencei as, for example, 
defined in ["20, Sec. 1] and fi?, Chap. 2]. Suppose that [A, B) 
are random variables on a discrete product space A x B 
with joint distribution pab- Let A. and B denote the n-fold 
Cartesian products of A and B respectively. The type of a in 
A. and the joint type of (a, 6) in ^ x B are the empirical 
distributions respectively defined by 



7r(a'|a) = 



|{z : Qi = a'}| 



a e A, 



and 

^{a ,h \a,b) ^ — {a ,b )eAy.B. 

' n 

Fix (5 > 0. The 5-typical, 5 -jointly-typical and 5-conditionally- 
typical sets are respectively defined by 

Ts{A) ^ |a e ^ : T:{a'\a)-pA{a') < SpA{a'), Va' G 
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Fig. 7. Word error rates of separate (solid lines) and joint decoding (dashed lines) as a function of the signal-to-noise ratio of the downlink channel. On the 
horizontal axis we also show the capacity of the downlink channel for easier comparison with the achievable regions. 



T5{A,B) ^ {(a, 6) eAxB: 

T:{a' ,b'\a,b) -pAB{a',b') < SpAB{a',b'), 

y{a\h') e X s}, 

and 

Ts{A,B\a) ^ {be B: (a, 6) e 7^(^,5)}, 

where pA denotes the ^-marginal of pab- We note that if 
(a, b) is in Ts{A,B), then a is in Ts{A) and b is in Ts{B). 
The next lemma will be used throughout the proof. 

Lemma 3 (Thms. 1.2 & 1.3, [20]): Fix 

< Si < $2 < min pab{cl, b). 

PAB{a.b)e support(pAB) 

If a belongs to Ts^ (A), then the cardinality of the conditionally 
typical set Ts2{A, B\a) satisfies 

Moreover, the probability that 

B = Bi,B2, . . . ,Bn 

(drawn i.i.d. with the B-marginal of pab) belongs to the 
conditionally-typical set satisfies 



)-2S2H{B)) 



3) Uplink Code Construction: Consider Node L Randomly 
partition the source space W*'' into 2"'' bins, labelled as 
{Bi' ,82 , ■ ■ • , .B*'' (,) }, using an i.i.d. uniform law; i.e.. 



1 



6'" e {1,2,..., 2"'-"'}. 



The allowable values of r"^ will be specified later. With a 
slight abuse of notation, let 

denote the map from source sequences to bin indices. Node I 
observes a sequence it?''' from W*'', and it sends the corre- 
sponding bin index 6''' ^ (it)*'' ) to the Relay. 

4) Downlink Code: Consider the Cartesian product set of 
bin indices, 

B^(g){l,2,...,2"'-"}. 

For each bin tuple b = . . . ,U^^) in 3, generate a 

downlink codeword 

a,'»'(b) = (xf(6),4°>(6),...,x<„°'(b)) 

by randomly drawing n-symbols from A"'"' using the marginal 
distribution of X^°K The Relay observes a bin tuple b = 
[U^\U^\ . . . , from B on the upUnk, and it sends a;<">(b) 
over the downlink broadcast channel. 

5) Decoding: Let denote the joint distribution of the 
source {W^^\W^^\ . . . , 1^<^'), and let 

n 
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Q(i)(w(i)) ^ |(6<'', 6<'', . . . , 6'^') for which there exists a unique tuple . . . , u;<^') 

such that w'-'^ belongs to S^,, , = 2, 3, . . . , L, and 

(-u;<^'><^',...,'ii;'^') e r5(M^™,...,VF<^V''')}- (H) 



Fix < (5 < where is the smallest value in the support 
sets of and the joint distribution {X''"\Y^^'). 

The decoding procedure used at each node is identical; we 
describe the procedure for Node 1 . The node observes a source 
sequence w'^' from W'^', it computes the corresponding bin 
index = 0'^' (ii;*^'), and it observes a channel output 
from y^^'' . The node compiles a list of bin tuples that contain 
unique source sequences (5-jointly typical with to this 

end, for each to'^' in W*^' let Q'^^{w"-^) be defined as in ( [TT| . 

In addition, the node looks for a unique tuple {IP^ , 6'^' , 
. . . ,U^^) in Q<^'(i(;<^') such that the corresponding broadcast 
channel codeword is (5-jointly typical with the observed chan- 
nel output, i.e.. 



(12) 



If successful. Node 1 sets (to' ■ \ ' i&* ' 
equal to (tt)*^' , iB*^' , iD*^' , . . . , w'^^^), where (iD*' 
■uj'^') is the unique vector identified by Q-'^\w'-'^'>) . If unsuc- 
cessful, the node declares an error. 

Remarks: 

• Noisy uplink: At the end of the proof, we will adapt 
the uplink code for noiseless channels to include noisy 
channels by simply adding a good point-to-point channel 
code for each orthogonal uplink. 

• Separation: Each node uses standalone source (and, later, 
channel) encoders on the uplink. Similarly, the Relay 
uses standalone channel decoders on the uplink and a 
standalone channel encoder on the downlink. However, 
the decoder at each node is a true JSC-decoder; for 
example. Node 1 first exploits its source side informa- 
tion lo'^' to compile the list (lo*^'), before decoding 
the transmitted codeword from Q-^\w'^^) using y'^\ It 
can be suboptimal to decode the transmitted codeword 
3.(0) 5(1) ^ 5(2)^ ^ using only the channel output y'^', 
which, for example, is the case in separate source-channel 
coding. 

6) Error Analysis: We wish to upper bound the average 
joint error probability P^, as defined in ([3]l, for the described 
ensemble of codes. By the union bound for probability, we 
have 



1=1 



(13) 



Consider the first error event of the sum ( [T3| ): 



i.e., the event that Node 1 decodes one or more source 
sequences in error. We now given an upper bound for P[£]. 
Fix 

0<S' <s 

arbitrarily. Consider the following events. 

(i) The event that the source sequences are not (^'-jointly 
typical: 

irs'{W\w\...,w''-')}. 

(ii) For each nonempty subset 

= {h,l2, ■ • ■ , ^|£|} 

of {2,3,..., L}, define the following event: the source 
sequences are (5'-jointly typical and there exists an \£\- 
tuple of (5-conditionally typical sequences, say 

= (t,;<'i',7i;('2),...,u;(VP), 
in the same bins as the source: 

such that w"^ ^ W^'\ (j)'"{w^'^) = (/)«(W^<"), e £}. 

(iii) The event that the broadcast channel input and output at 
Node 1 are not (5' -jointly typical: 

£3 ^ { (X'"' {B'» , B'^) , . . . , B*^' ) , F<^' ) 

(iv) The source sequences are (^'-jointly typical, the channel 
input and output are (5'-jointly typical, and there exists 
another ^-conditionally typical codeword with bin in- 
dices in Q»{W): 

£^ A ei n £1 n ^(\P\~}f'\ . . . , &<^') e Q«(Ty«) 

such that (6<^', 6'-'', . . . , 6<^>) ^ {B^'\B''\. . . , B<^') 
and a;'"* (B<^' , 6<^> , , . . . , 6'^' ) e Ts (X*"' , \ F'^' ) } . 

The error event £ — the event that Node 1 decodes a source 
sequence in error — is a subset of the union of £1, U£2.c^ ^3 
and £4, hence, 

P[£] < P[£i] + 'P[^2,£] +P[£3] +P[£4]- (14) 

£C{2,3,...,L} 

It follows from the law of large numbers that 

limP[£i]=0 and lim P[£3] = 
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(/3) 



(7) 
(7) 

(b) 



(7) 



< 2 



(15) 



e.g., see |[T4| Sec. 2] or ||20) Thm. 1.1]. 

The probability of each can be upper bound as shown 
in ( [T5| ), where 

• 1{} denotes the indicator function. 

• The sum marked with (a) is taken over all 

{w'^'\w'-'\ . . . ,t(;<^') e W''' X W"' X • • • X W'"-'. 

• The sums marked with {(3) are taken over all 

with 0(u;<") 7^ (/>(i(;<") for all / € £. 

• the sums marked with (7) are take over all 

• Equality (a) follows because the probability that each 
sequence in W*'' is randomly assigned to the same 
bin as it)*'' is independent of all other bin assignments 
and equal to 2^"''*'' . 

• Inequality (b) bounds the cardinality of of the condition- 
ally typical set Ts (VK<^> , W^p> , . . . , W^<^' \w'» , w'^"^ ) using 
Lemma |3] 

Finally, we notice that if 



(a) We let 



H{W'^'>\W''\W"''^)il + S) < ^1 



(16) 



then 



lim V[£2,c] = 0. 



The probability of £4 can be upper bound as shown in ( [T7] i. 

• The sum marked with (a) is taken over all 

{w'^'\w'-'\ . . . ,t(;<^') e X W"' X • • • X W'"-'. 

• The sums marked with {(3) are taken over all 

(6<=',6P>,...,6'^') e Q«(i(;«). 

• The sums marked with (7) are taken over all 

(-u;<^' , w'^-> , . . . , lo*^' ) e (M^'^' , VF'^' , . . . , W^<^' ) . 

The reasoning behind (in)equalities (a) through (d) is as 
follows. 



for Z = 1, 2, . . . , L, denote the bin index of the Z-th source 
sequence, 
(b) We have that 



implies 



(c) We bound the probability that an alternate codeword, 

P>, P', . . . , 6<^'), is (5-jointly typical with y'^', 
using Lemma [3] and the fact that the rt-symbols of the 
codeword are drawn i.i.d. with the marginal of X'"'. 

(d) The cardinality of Q''^\w^^') must be smaller than the 
cardinality of the conditionally typical set Ts{W^^\ W'-^\ 
. . . , W^'jii;*^'), which in turn is smaller than the bound 
of Lemma |3] 

Finally, it follows that 



lim ] 

n— J-oo 



whenever 



- 6{H{W''\W''\ . . . , VF'^'IW'™) + 2H{X^">)). (18) 

The above analysis can be repeated for each of the L nodes 
to obtain constraints analogous to ( [T6] l and ([T8|. We are free 
to choose 6' and S arbitrarily small, so Pg (averaged over the 
ensemble of codes) can be made arbitrarily small by increasing 
n if 

F(W<^'|W^<^'') <^r», (19) 
lec 

holds for each nonempty and strict subset £ of {1,2,..., L} 
and 



(20) 



holds for each I — 1, 2, . . . , L. The random coding argument 
implies that there must exist at least one code from the 
ensemble with an error probability at least as small as Pg. 
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(a) 



^ P [Y'» = I ( W-'' , W-'' W'^^ ) = («;<^' , - 
yWeyw 

• 1 { (X<"' (6« , . . . , 6<"' ) , y« ) e Ti' (X<"> , r« ) } 



(0) 



(b) 



IDI , 

< ^q,(^<;<^«;'^...,^l;<->) 



(7) 



ymers,{Ym} 



w 



(7) 



y(i)gr,,(y(i)) 



(d) , 

< ^q,(«;<^«;'^..., 

(7) 

^ P = I ( Vr<^' , VF'^' , . . . , W^<^> ) = («;<^' , w''^ , . . . , -u;'^' )] 



(17) 



7j Extension to Noisy Uplink Channels: Consider the setup 
of Theorem [T] with noisy (orthogonal) uplink channels. The 
random-coding argument of Sections A-A3 to A-A5 can be 
extended to this setting by the use of a separate source 
and channel code architecture on the uplink. That is, use a 
(point-to-point) capacity-approaching code for each orthogonal 
uplink channel, choose the source-code compression rates to 
match the channel coding rates (r"' = C*'^ — C,, where C is 
sufficiently small), and communicate the L bin indices to the 
relay using the point-to-point channel codes. ■ 



B. Proof of Theorem [7] ( Converse) 

Suppose that we have a JSC-code with < e. The next 
two lemmas will be useful. 

Lemma 4: For each nonempty and strict subset £ of 
{1, 2, . . . , L}, we have 



1 



Proof: Choose I from arbitrarily. We have 



(a) 1 
< - 

n 



< 



[/i(e) + enlog|W"| 

L h{e) 



eL max |W"'' 



where ein, e) as e ^ 



where h{e) is the binary entropy function and (a) follows from 
Fano's inequaUty. ■ 

Lemma 5: For each nonempty and strict subset C of {1,2, 
. . . , L}, we have 

Proof We have 

^ j(x(o.y(o,o) ^ ^H{Y^°''^) - F(y<°'"|x®; 

> HiY'"'''') -Y,H{Y'-"''^X'^'>) 



lec 
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where (a) is a consequence of the Markov chain 



Proof of Theorem |7] f Converse): For each nonempty and 
strict subset £ of {1, 2, ... , L}, we can lower bound the right 
hand side of (Hal by 



-/(X<'>;1^<'">) 



(b) 1 



> -/(X'^';Y<°-^') 
n 

n 



1 



(d) 1 



> 



1 r 



(21) 



where (a) follows from the definition of channel capacity; 
(b) follows from Lemma [5] (c) follows from the Markov 
chain (Ty<^', VF*''"')^^*'"^^"' '"; (d) follows from the 
Markov chain W^'''>^[W''''''\Y^"-''^)^Y^''''^; and (e) fol- 
lows from Lemma m and that the source is i.i.d. 



Consider ( |4b[ l. Let p^io) denote the pmf of the i-th symbol, 



Xf^, of X'°\ Define a new time-averaged random variable 
X°'> on A""" via the pmf 

1 " 

j=i 

For each Hn {1, 2, . . . , L}, we have 

(''11 ^ 

J(^(0).y(0) > i^/(xf 
i=l 

(b) 1 

> -/(X<">:F<") 
n 

n 

> -/(Ty«"^';r<"|M^<") 



> i?(W"'>^'|iy<")-e(n,e), 



(22) 



where (a) follows from Jensen's inequality and the fact that 
is concave in Px{o)', (b) follows because the 
broadcast channel is stationary and memoryless; (c) follows 
from the Mai'kov chain (VF<'>, T^"'^'>)^^*°*^^"'; and 
(d) follows from Lemma [4] 

Consider a sequence {e} — 0. For each e in this sequence, 
we have by definition at JSC-code for which ( |2T| and 
hold for some X'-°^ on X'-°\ The proof is completed by noting 



that the sequence of pmfs p ^(o) will converge to some pmf on 
A"'"'. " ■ 

Appendix B 
Counterexample 

We now describe a situation where the achievability as- 
sertion of Theorem [T] holds, but that of Theorem |2] fails; 
equivalently, reliable communication is achievable with JSC- 
codes of the form (|2]l, and it is not achievable with separate 
source and channel codes of the form ^ and (|7]l. 

Example 1: Suppose we have 3-nodes with the following 
sources: 



W^'^ =U2® Ui2 ® U23 
= C/3 ® [/i3 ® U23, 



(23) 
(24) 
(25) 



where each Ui G {0, 1} is an independent random variable, for 
all i G {1, 2, 3, 12, 13, 23}, and ® denotes the XOR function. 
We choose Pr{Ui = 1} = 0.0085, Pr{U2 = 1} = Pr{C/3 = 
1} = 0.0052, Pr{L/i2 = 1} = Pr{f/i3 = 1} = 0.0128, and 
Pr{C/23 = 1} = 0.138. For these choices of probability mass 
functions, we have 

= iJ(M/p)|M/w,VF<^)) 

= 0.10 (26a) 

= 0.30 (26b) 
H{W'^\ = 0.70 (26c) 

= 0.21. (26d) 

The information diagram |13, Sec. 3.5] of such a source is 
depicted in Fig. [8] 

Suppose the uplink channels are point-to-point channels 



with capacities CJ,p' 



2, and the downlink 



channels are r« = X'-°'> ® iV«, for each i € {1,2,3}, 
where each X**' e {0, 1}, and each iV<'' is an indepen- 
dent random variable. We choose Pr{A^<^' = 1} = 0.0508 
and Pr{A^(") = 1} = PT{m'^ = 1} = 0.184. Note 
that the uniform input distribution simultaneously maximises 
j(j^(o).y(.)) for all i g {1,2,3}, giving /(X<'»;ym) = 0.71 
and /(X("';y<"') = /(XC';^'^') = 0.31. These assumptions 
satisfy the achievability requirements of Theorem [T] 

We now show, via a contradiction, that there does not exists 
a rate tuple (r'^', r<^', r<^') satisfying ([8|. Suppose there exists 
(r«,r<"',r<^') such that 

0.10 = H{W^'^\W'-^\W'-'^) < r'^' < CJ,« = 2 (27a) 
0.10 = ir( I W-" , T^P' ) < r<"' < C^p' = 2 (27b) 
0.10 = Ty<=') < rP' < Cf,' = 2 (27c) 



0.30 = H{W'»,W''^\W"'>) < + r<^> < C« + C'^' = 4 

(27d) 

0.70 = i/(W^<^', W^<^>|W^W) < r<^' + r'^'> < O^l + Cf,' = 4 



(27e) 
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Fig. 8. The information diagram of (VK<^ , , VF<^') defined in (26) . 
Eacli of the three circles represents a source message, and the intersection 
between two circles represent the mutual information between the sources. 
The numbers are the values of the corresponding conditional entropies and 
mutual informations. 



0.30 
and 



(27f) 

(27g) 
(27h) 
(27i) 



r«+r<^' </(X<°>;r<^') =0.31 
^(2)^^(3) </(x(°);r«) =0.71 
r«+rP' </(X<''>;r<^') =0.31. 
The left inequalities in ( |27a| i and \21e\ imply that 

max{rW + r<'', r<'' + r<''} > 0.45. 
However, ( 27g i and \21'\) together imply 

max{rW + r<=', r'" + r<''} < 0.31, 
which gives the desired contradiction. 
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